Assessment of Group and Individual Learning through
Intelligent Visualization Workshop (AGILeViz)

CSCL 2007

Eric Hamilton and Andrew Hurford

Workshop Committee:

Amy Baylor, Mark Chavez, Ron Cole, Chris DiGiano, Michael Jacobson, Manu Kapur, Leo Lee, Richard Lesh, Manuel Lima, Philip Vahey 1

Purpose of the Workshop: This workshop will present, discuss, and explore new ways of providing educators with sophisticated, computer-generated visualizations of learning by groups and individuals in order to improve teaching and learning. The organizers of the workshop are motivated by the convergence of several factors that bear directly on the CSCL vision to understand and encourage novel, technology-rich collaborative learning:

• Educational innovators are using technology to understand and propel learning in ways that dwarf the limited paradigms of learning and assessment that dominate educational practice.
• Current assessment tools not only are highly constrained in the constructs that they measure, but the representational systems used to communicate student progress on these assessments are also very limited. Representational systems used are inadequate for supporting thinking about the kinds of learning and problem-solving behaviors that are increasingly important to assure.
• Interactive visualization techniques for large quantities of heterogeneous data have now reached a state of maturity that make them ripe for applications in data-driven analysis and decision making for education.

This workshop is based on a conjecture that complex visualization systems may be applied to analyses of learning and in support of learning to create a new class of tools and environments for promoting collaborative learning and learning research.

Discussion: Our goal is to nurture the nascent area of applying complex graphical systems for analysis and assessment of complex individual and group learning. This area is ripe to become a “hot topic” and we believe that this workshop will likely lead to ground-breaking subsequent literature that will prove crucial to educational innovation. The lead organizer of this workshop is a former division director at NSF and another member of the program committee (Amy Baylor, participating in an independent researcher capacity only) is a current rotator at NSF. We are confident, based on our collective experience, that this is the sort of promising area that can lead research agencies in the US and abroad to create new programs and opportunities for research funding.

Some of the most intractable challenges in education policy and practice involve effective design and use of tools to assess learner progress. Current assessment tools typically produce paltry amounts of one-dimensional information from which large and important inferences are drawn about learning. While progress has been made in quality and dimensionality of assessments (e.g., progress maps, NRC 2001, NRC 2006), the majority of data collection is still very labor-intensive and representations are typically limited to charts and graphs, spreadsheets, or single number scores. We intend to begin conversations and collaborations that will expand the “assessment triangle” into a third dimension, adding another of the same type of co-contributing dimension as Pellegrino et al (NRC, 2001, pp. 44-51) intended for the original three: observation, interpretation, and cognition combined enable meaningful assessment. We would like to add high-tech and novel data capture and representation as another dimension of analysis.

We believe that a new era has arrived in which technological tools, thinking about learning, and ways of representing and describing learning are co-evolving into dynamic new forms of research and understandings of teaching and learning. Space does not allow discussion, but consider a continuum of learning data collection systems (e.g., progress maps, NRC 2006, p. 78), representation systems (learning progress maps, Lesh, et al., 2002), and computer-supported learning environments (ALASKA, Hamilton, 2005; flight simulators with six pilots flying against simulated intelligent enemies : Participatory Simulations, Wilensky & Stroup, 1999; and MMORPGs, Squire, 2006). Computer driven data collection and representation are providing teachers, students, and researchers with innovative and powerful new “sightlines” for perceiving learning and teaching and for making moment-to-moment decisions about progress and how best to proceed.

Workshop Description: This half-day workshop will feature presentations by the committee on the state of the art in their respective research and will respond to questions from the audience. After four presentations outlining the scope of possibilities in innovative assessment and representation schemes, the participants and committee will form small groups brainstorming and collecting ideas, suggestions, and questions.

 Products: Most importantly, we hope to encourage collaborations between a diverse group of attendees and to stimulate the growth of research, understanding, and utilization of new technological tools and ways of thinking.
________________________________
1 Short biographies of the committee members, references for citations, requests for papers, and more information can be found at http://agileviz.net